Papers with fine-tunes open-sourced LLMs
DIRAS: Efficient LLM Annotation of Document Relevance for Retrieval Augmented Generation (2025.naacl-long)
Copied to clipboard
| Challenge: | RAG systems leave out important relevant information (low recall) and excessively related but irrelevant information (high precision) authors propose a manual annotation-free schema that can be used for RAGs with limited performance. |
| Approach: | They propose a manual annotation-free schema that annotates unseen queries with calibrated relevance scores. |
| Outcome: | Evaluators show that DIRAS can achieve GPT-4-level performance on annotating and ranking unseen (query, document) pairs. |